Convex Foundations for Generalized MaxEnt Models

We present an approach to maximum entropy models that highlights the convex geometry and duality of GEFs and their connection to Bregman divergences. Using our framework, we are able to resolve a puzzling aspect of the bijection of [1] between classical exponential families and what they call regular Bregman divergences. Their regularity condition rules out all but Bregman divergences generated from log-convex generators. We recover their bijection and show that a much broader class of divergences correspond to GEFs via two key observations: 1) Like classical exponential families, GEFs have a “cumulant” C whose subdifferential contains the mean: Eo∼pθ [φ(o)] ∈ ∂C(θ); 2) Generalized relative entropy is a C-Bregman divergence between parameters: DF (pθ , pθ ′) = DC(θ ,θ ′), where DF becomes the KL divergence for F = −H. We also show that every incomplete market with cost function C (see [2]) can be expressed as a complete market, where the prices are constrained to be a GEF with cumulant C. This provides an entirely new interpretation of prediction markets, relating their design back to the principle of maximum entropy.